Table of Contents

LCS-SA Project: Vanderbijilpark, South Africa 2021.

Objectives

Aim:

  1. Environmental health in sub-Saharan-Africa - leveraging local and global air pollution data for epidemiological research (LEAP-Epi)

Objectives:

  1. Determine the precision, accuracy, reliability, rigidity and usability within a South African setting.
  2. Provide information which could be used to improve the low- and medium-cost sensors.

Specific research questions:

  1. XXX
  2. XXX

Libraries, Packages and General Functions

Back to the top

Import Libraries

Google Sheets for Quality Control

Admin functions

Directory pathways

Back to the top

Level 1 - Import Raw Data Files

Back to the top

Action to take place before Level 1 data:

  1. Import Level 0 raw data files.
  2. Format into consistent datasets by combining individual data files.
  3. Apply missing values.
  4. Remove variables that will not be used in the final dataset.

00_Reference_SAWS_df0

Back to the top

Look at the files

Import the raw data

Format values

Correct values according to status

Look at the raw data

Quality Control

NO

Adjust baseline negative values

NO2

Adjust negative baseline

NOx

Adjust baseline

SO2

Correct instrument drift

CO

Correct baseline drift

01_ES642_U16486_df1

Back to the top

Function to read files

Read files

Set timezone to UTC

Convert to SAST

Remove data where flow is less than 1.8 and more than 2.2

Remove dates with unrealistic data

Quick Look

Save to CSV

02_ES642_U16489_df2

Back to the top

Import data

Convert timezone to SAST

Remove data where flow is less than 1.8 and more than 2.2

Remove unrealistic data

Quick Look

Save to CSV

03_ARISense_SN000-57_df3

Back to the top

Timezone : local

Define variable names

Import Data

Define columns

QC

Quick Look

Saving the data files

04_ARISense_SN000-59_df4

Back to the top

Timezone: local

Data file pathway

Dataframe: info, head, tail, descriptive statisitcs

Define Columns

Quickview plot of the available data

QC

Saving the data files

05_Vaisala_S1830003_df5

Back to the top

Data file pathway

Files to read

Combine

Quick View

Saving the data files

06_S500_5002-2D82-001_df6

Back to the top

Import Data

Quickview plot of the available data

Saving the data files

07_S500_ECM-1906191-003_df7

Back to the top

Define variable names

Import data

Remove unused

Quick View

Saving the data files

08_PolludroneSmart_EA01P0001_df8

Back to the top

Import data

Converting the units to the correct format

Remove unused variables

Quickview plot of the available data

Saving the data files

09_SimplicityV1_CCSENV020_df9

Back to the top

Import data

Remove unused variables

QC

Saving the data files

10_SimplicityV1_CCSENV011_df10\

Back to the top

Import data

Remove unused variables

Quickview plot of the available data

QC

Saving the data files

11_SimplicityV2_IMTAQS0001_df11

Back to the top

Define variable names

Import data

Remove unused variables

QC

Quick view

Saving the data files

12_SimplicityV2_IMTAQS0002_df12

Back to the top

Define variable names

Import data

Remove unused variables

Quickview plot of the available data

QC

Saving the data files

13_SimplicityV2_IMTAQS0003_df13

Back to the top

Define variable names

Import data

Remove unused variables

Quickview plot of the available data

QC

Saving the data files

14_SimplicityV2_IMTAQS0004_df14

Back to the top

Define variable names

Import data

Remove unused variables

Quickview plot of the available data

Saving the data files

15_ECOMSMART_20149_df15

Back to the top

Import data

Remove unused variables

Quick look

Saving the data files

16_RAMP_173_df16

Back to the top

Define variable names

Data file pathway

Dataframe: info, head, tail, descriptive statisitcs

Quickview plot of the available data

Remove unused variables

Quickview plot of the available data

Saving the data files

17_GM-5000_CM21035290_df17

Back to the top

Import data

Set limits

Quick look

Save

18_Plantower_xxx_df18

Back to the top

Import Data

Quickview plot of the available data

Quickview plot of the available data

Rename

Saving the data files

19_Plantower_xxx_df19

Back to the top

Define variable names

Data file pathway

Dataframe: info, head, tail, descriptive statisitcs

Quickview plot of the available data

Remove unused variables

Quickview plot of the available data

Saving the data files

20_Zephyr_642-SA_df20

Back to the top

Data file pathway

Import data

Remove unused variables

Quality control

Quick view

Saving the data files

21_Zephyr_729-SA_df21

Back to the top

Data file pathway

Import data

Remove unused variables

Quality control

Quickview plot of the available data

Saving the data files

22_Atmos_84CCA8B167D2_df22

Back to the top

Import data

Quickview plot of the available data

Saving the data files

23_Atmos_98F4ABDCA328_df23

Back to the top

Import data

Quickview plot of the available data

Saving the data files

24_Dylos_xxx_df24

Back to the top

Import data

Quickview plot of the available data

Saving the data files

26_Zephyr_533_df26

Back to the top

Data file pathway

Import data

Quick plot

Save

27_PolludroneSmart_PM01P0007_df27

Back to the top

Import data

Quick Plot

Save

28_RAMP_177_df28

Back to the top

Level 2 - Combined Dataframes

Back to the top

Here all the data from the individual instruments are combined into a single dataset

Level 3 - Data Quality Control

Back to top

Action to take place before Level 2 data. Flags for QC are defined as follow:

  1. Instrument failure or not present
  2. Power failure (Not applicable to this project as Line Voltage was not monitored)
  3. Variable below detection limit
  4. Out of instrument range
  5. Stuck value
  6. Flagged for out of climatology

Level 4 - Data analysis

Back to the top

  1. xxx
  2. xxx
  3. xxx
  4. xxx
  5. xxx
  6. xxx

Correlation between reference instrument and low-cost instruments

Timeseries

Diurnal plots

Data recovery

Correction Factors for low-cost instruments

Reference air quality

Diurnal Variability